Execution engine for executing single assignment programs with affine dependencies

ABSTRACT

The execution engine is a new organization for a digital data processing apparatus, suitable for highly parallel execution of structured fine-grain parallel computations. The execution engine includes a memory for storing data and a domain flow program, a controller for requesting the domain flow program from the memory, and further for translating the program into programming information, a processor fabric for processing the domain flow programming information and a crossbar for sending tokens and the programming information to the processor fabric.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part application of co-pendingU.S. patent application Ser. No. 12/467,485, filed May 18, 2009, andtitled, “EXECUTION ENGINE FOR EXECUTING SINGLE ASSIGNMENT PROGRAMS WITHAFFINE DEPENDENCIES” which claims the benefit of U.S. Provisional PatentApplication Ser. No. 61/130,114, filed May 27, 2008 and titled“EXECUTION ENGINE”; which are both hereby incorporated by reference intheir entireties for all purposes.

FIELD OF THE INVENTION

The present invention relates to the field of computer systems, and inparticular to the efficient execution of fine-grained parallelinstructions.

BACKGROUND OF THE INVENTION

A typical general purpose computer is configured as a sequentialinstruction stream processor, which fetches instructions from memory,decodes, and executes these instructions. The sequential instructionstream processors use energy very inefficiently with more energyconsumed in the instruction management than in the actual execution ofthe operation that the instruction represents. For example, moderngeneral purpose x86 processors from Intel or AMD only attain 10% of peakperformance as measured by the operational throughput of the processoron important algorithms such as sparse matrix solvers.

Furthermore, these sequential instruction stream processors are veryinefficient for fine-grained parallel computation. In the aforementionedsparse matrix solver, performance requirements typically require thatthousands of processors are used concurrently. To coordinate executionamong groups of processors, much time and energy is wasted when someprocessors finish before others and subsequently need to wait tosynchronize with the rest of the processors.

The algorithms for which the general purpose computer is becoming lessand less efficient are of vital importance to science, engineering, andbusiness. Furthermore, the exponential growth of data and computationalrequirements dictates that groups of processors are used to attainresults in a reasonable amount of time. Many of the important algorithmssuch as signal processing, solvers, statistics, and data mining, exhibitfine-grained parallel structure. Mapping these algorithms on networks ofgeneral purpose processors is becoming problematic in terms of size,cost, and power consumption.

SUMMARY OF THE INVENTION

The present invention is an apparatus for the efficient execution ofhighly parallel fine-grain structured computations. The apparatus isprogrammable to perform efficient execution on a wide variety of suchstructured computations. Energy consumption in a computer isproportional to the number of instructions executed and data operandsneeded. In an electronic implementation of a computer, this implies thatenergy consumption is proportional to time and distance instructions anddata need to travel. This invention generates a physical model ofexecution that honors spatial distances and the invention organizes thecomputation in such a way that contention among instructions is managedthrough a simple queuing system.

An apparatus in accordance with the present invention includes aprocessing element that receives data tokens from a fabric of processingelements and matches these to instruction tokens on the basis of aspatial tag. This tag creates spatial relationships between computationsto reflect energy and temporal optimizations the algorithmic designerintended.

In some embodiments, the processing element fabric is supplied datatokens from a bank of data streamers, which transform data structuresthat are stored in computer memory into a stream of data tokens.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention may be understood with reference to the followingdrawings in which like elements are indicated by like numbers. Thesedrawings are provided to illustrate selected embodiments of the presentinvention and are not intended to limit the scope of the invention.

FIG. 1 is a block diagram of a generalized data flow computer systemthat operates according to some embodiments.

FIG. 2 is a block diagram of a data streamer according to someembodiments.

FIG. 3 is a block diagram of the processor fabric according to someembodiments.

FIG. 4 is a block diagram of the processing element according to someembodiments.

FIG. 5A demonstrates one particular embodiment of a data packet.

FIG. 5B demonstrates one particular embodiment of a data token.

FIG. 5C demonstrates one particular embodiment of the instruction token.

FIG. 6 . is a block diagram of one particular embodiment of theinstruction store FIG. 7A is an example of a single assignment formprogram for matrix-vector multiplication.

FIG. 7B shows the associated single assignment graph in two dimensions.

FIG. 8 illustrates a flowchart of a method of utilizing an executionengine.

DETAILED DISCUSSION OF THE INVENTION

The following discussion sets forth numerous specific details to providea thorough understanding of the invention. Those of ordinary skill inthe art having the benefit of this disclosure will appreciate that theinvention may be practiced without these specific details. Various wellknown methods, procedures, components, and circuits have not beendescribed in detail in order to focus attention on the features of thepresent invention.

An execution engine executes single assignment programs with affinedependencies. Programs in single assignment form (SAF) are algorithmsthat express the computation as a set of equations where each assignmentrefers to a unique identifier. Typical expressions of single assignmentprograms are recurrence equations where the left hand side isinterpreted as a unique identifier. Many algorithms have naturalexpressions in single assignment form. For example, FIG. 7A shows anexample of a single assignment program for matrix-vector multiplication.FIG. 7B demonstrates the associated single assignment graph, which is adata flow graph where nodes represent computations and edges representdata movement.

A skilled operator in the field understands that single assignmentprograms and the terms “dependence graph” and “reduced dependence graph”are interchangeable. Furthermore, in the compiler literature, thedefinition of a control flow graph is well established. A control flowgraph represents all paths that might be traversed through a programduring its execution. Thus, if a program includes a collection of singleassignment programs, they may communicate their inputs and outputsdirectly or indirectly. As a compiler analyzes such a program, itrepresents this knowledge in a control flow graph. Again, a skilledoperator in the field understands that a program can be represented by acontrol flow graph and that the terms represent the same entity.

Since the engine executes a collection of single assignment programscommunicating potentially through a Random Access Memory, then by theinterchangeable nature of the terms “single assignment program” and“reduced dependency graph,” and the a priori knowledge that a program isable to be represented by a control flow graph, then it logically flowsthat a “domain flow program” is able to be defined as “the reduceddependency graph of a set of coupled affine recurrence equations plusthe control graph that couples dependency graphs that originate andterminate in the Random Access Memory.”

As described herein, the “program” is described as a “domain flowprogram” that represents the computation through dependency graphs anddomains of computation. From the operation of the machine, it is clearthat this machine executes on the basis of the data flow executionmodel, not the stored program model. It differs from the traditionaldata flow machine by the “program” representation, which includesspatial tags that are interpreted as points in an abstractmulti-dimensional lattice. What makes a program a “domain flow” programis the representation of the “fine-grain structured” computation interms of dependency graphs and domains of computation.

Additionally, described herein is that the recurrence equations, and byequivalence, single assignment programs, are valid only on their ‘domainof computation’. As the fabric has a collection of these domains activeduring operation, and the input of these domains ‘flow’ from a RandomAccess Memory, through the fabric, to the Random Access Memory, thisdescribes a flow of domains, or otherwise stated, a domain flow. Theprogram that represents this domain flow can thus be identified as adomain flow program that includes a set of coupled affine recurrenceequations, which can be represented by reduced dependency graphs, withinput domains that originate in Random Access Memory and terminate inRandom Access Memory. The description of such flows are importantcomponents in compilers and the compiler literature uses the term‘control flow graphs’ to refer to these descriptions. Logically, thesedescriptions introduce the concepts of a domain flow program as a set ofcoupled affine recurrence equations that the literature equates withdependency graphs or reduced dependency graphs, and that are representedby a control flow graph that describes the coupling of these singleassignment programs and how these domain flows originate and terminatein Random Access Memory.

The same way a program is able to include a collection of subprograms, asingle assignment program is able to include one or more singleassignment subprograms. By applying this subdivision on the domain flowprogram recursively, it ends up with the nodes in the control flowgraph, each equivalent to some set of affine recurrence equationsdefined over a domain of computation. As described herein, data tokensbelonging to different recurrence equations are organized by theiridentifier, their domain of computation, a signature update function,and a routing vector. Thus, the organization of data tokens in terms ofcommunicating affine recurrence equations that can be succinctlydescribed by a domain flow program is described including the concept ofdata tokens belonging to the same single assignment equation as beingdisambiguated by their identifier and their index in the domain ofcomputation. Also described herein is the process of one or moreprocessing elements producing output streams, which are series of datatokens that belong to the same recurrence equation disambiguated by anidentifier and defined over a domain of computation.

In FIG. 7A, an example is given of a system of recurrence equations,which is equivalent to a coupled set of affine recurrence equations asthe recurrence equation defining the w recurrence is dependent, orcoupled, with the recurrence equation defining the v recurrence.Furthermore, the notion of dependence graphs to represent these systemsof recurrence equations is to capture the coupling between therecurrence equations. If these equations were not coupled the dependencegraph would include only of nodes representing the single assignmentrecurrences.

Any program is able to be expressed in single assignment form. Programswith fine-grain structured parallelism are most naturally expressed insingle assignment form because single assignment form algorithms aremaximally parallel formulations that do not have any explicit sequencingto schedule execution. This allows the execution environment to focus onhow to manage resource contention. A program in single assignment formcontains a unique assignment for each operation. This is able to berepresented by a single assignment graph (SAG) that contains a node forevery operation and an edge for every data movement. A computationalevent is defined as some operation taking right hand side operands andcomputing the assignment on the left hand side of the equation.

Because of the limited speed of signal propagation, any two concurrentcomputational events are separated either in time or in space. Byembedding a single assignment form program in an abstract lattice,defined as a discrete subgroup of RN that spans it as a real vectorspace, an algorithm designer is able to specify spatial distance betweendependent computations. In some embodiments, the embedding isaccomplished in an abstract orthonormal lattice. An orthonormal latticeis defined by an orthonormal basis; all basis vectors have unit length,and their inner products are zero. The single assignment graph is ableto be embedded in the lattice with the rule that no dependentcomputational events are able to reside at the same lattice point. Thiswill assign each computational event a unique location in the latticeand separate dependent computations by physically separated latticepoints, thus making explicit the temporal separation for communicatingthe dependent operands. This unique location in the lattice is calledthe signature of the computational event, and it is defined as the indexvector identifying the lattice point in the lattice. Other examples ofappropriate lattices would be crystal groups and other discretesamplings of space that are defined by some regular cell that generatesa cover of space. There are well defined mathematical structuresdescribing these samplings called lattice groups. Orthonormal latticesare a subset of these more general lattice groups.

After embedding the SAG in some lattice, all program inputs and allcomputational events have an explicit routing vector that defines howresults are delivered to the dependent computational events. Thisrouting vector is called the dependency vector since it specifies howcomputational instructions depend on their inputs. The embedding in aspatial lattice allows the algorithm designer to incorporate constraintssuch as distance and resource contention. Distance is very important forpower efficiency and performance since both energy consumption and timeto communicate a dependent operand is directly proportional to distance.Distance here is defined in terms of hops in a discrete routing networkso each hop needs to go through at least one register thus consumingenergy proportionally to the number of hops.

The execution engine described herein is an efficient execution enginefor above-mentioned embedded single assignment programs embedded in anabstract spatial lattice.

Referring first to FIG. 1 , there is shown a block diagram of ageneralized data flow computer system that operates according to someembodiments. The computer system 100 includes a memory 110 that containsthe data and the program to execute. The memory 110 is able to be anyapplicable type of memory. Execution starts by the controller 120requesting a program from the memory 110. The controller 120 presents aread request via bus 121 to the memory controller 130 which translatesthe read request to a memory request and returns the data to thecontroller 120. This data contains the program instructions to execute asingle assignment program. The controller 120 decodes these programinstructions and translates them into programming information for thestreamers 140, and are delivered to the streamers 140 via a control bus122. Concurrently, the controller 120 sends programming information forthe processor fabric 160 through a command bus 123 to the crossbar 150.The crossbar 150 delivers the programming information including commandsto the proper rows and columns of the processor fabric 160 which readsthese commands and configures itself to execute the single assignmentprogram. Once the overall data path is set up, the controller 120 issuesan event to the streamers 140 to start. The memory controller 130receives read and write requests from the streamers 140 and transformsthem into memory requests. On read requests from the streamers 140, datais delivered by the memory controller 130 to the streamers 140 andforwarded to the crossbar 150. The crossbar 150 routes the data streamsto the appropriate rows or columns in the processor fabric 160. Theprocessor fabric 160 receives the incoming data streams, executesinstructions on these streams and produces output data streams. Theseoutput data streams are written back to memory 110 by traversing thecrossbar 150 to the streamers 140 that associate memory addresses to thedata streams, and then present them to the memory controller 130, whichwill write the data streams into memory 110. Once the streamers identifycompletion of the computational task, notifier events are sent back tothe controller 120 via the control bus 122. Command/event buses 123 and124 are used to communicate interrupts and other events back to thecontroller 120 to signal errors, panics, and operational informationthat the controller is able to use to interrupt or otherwise steer thecomputation.

Referring now to FIG. 2 , there is shown a block diagram of a datastreamer 140. A data streamer executes a stream program that assemblesand/or disassembles a data token stream. The process starts by thecontroller 120 decoding a single assignment program and transformingthis program into stream programs for the streamers 140. The controller120 writes this stream program to the stream program store 220 via acontrol bus 122. A stream program details how to calculate requestaddresses and other attributes such as size and type. The stream programis executed by the processor 210. The processor 210 executes the streamprogram and produces memory controller commands which are written intothe command data queue 250. As part of the same stream program,processor 210 also produces the associated token attributes recurrenceidentifier, signature, and data type. Those attributes are consumed bythe token assembly unit 230, which combines these attributes with readdata 260 that is coming from the memory controller 130, to produce adata token stream that is written to the cross bar 150. Similarly, fordata streams that come from the crossbar 150 and that need to be writtenback to memory, the token disassembly unit 240 strips the tokenattributes recurrence identifier, signature, and data type from theincoming data token and writes them in the stream program store 220 forfurther processing by the processor 210. The token disassembly unit 240writes the data payload of the data token into the write data queue 270.Concurrently, the processor 210 executes a write stream program togenerate the proper memory write commands which are, with the writedata, presented to the memory controller 130.

Referring now to FIG. 3 , the crossbar 150 provides an interconnectionnetwork to connect N data streamers to M processing elements (PE) 310 inthe processor array. The streamers 140 transform a flat data structurein memory into a multidimensional data stream in time. The crossbar 150connects the physical position of any streamer to one, many, or allphysical positions of the processing elements in the processor fabric160. The processor fabric 160 consumes these multidimensional datastreams and connects them together inside the fabric under the controlof the single assignment program that controller 120 read from memoryand installed in the processing elements of the processor fabric throughan instruction token stream written on bus 123 that uses the crossbar150 to connect to the appropriate processing elements 310. Themultidimensional data streams flow through the processing elements 310and the processing element routing network 320. The PEs 310 processinstruction and data tokens. Instruction and data tokens are able to berouted and stored in the PE 310. The single assignment program describeshow data tokens combine in some multi-dimensional space. The PEs 310recognize the spatial tag called the signature of a computational eventand take action under control of the single assignment program installedin their program store by controller 120. The action is to compare thesignature of an incoming data token with the signatures of instructiontokens that accumulate in the PEs 310 during execution, and extract thedata token from the internal network 320 if there is a match. Theoverall computation represented by the single assignment program evolvesas the multi-dimensional data streams match up within the processingelements 310 and produce potentially new multi-dimensional data streams,representing intermediate results, which are propagated through therouting network 320 to their computational event destination in theprocessor fabric. Final result data streams are streamed back to thecrossbar 150 via the routing network 320 and the PEs 310 under thecontrol of the single assignment program stored in the PEs 310. Thecrossbar 150 presents these data streams back to the appropriatestreamers 140 which transform them into flat data structures again thatare able to be written to memory 110. During the execution of the singleassignment program error conditions are able to occur, or it may be ofinterest to observe processing events so that the controller 120 is ableto better manage the computation. Examples of program errors arefloating point exceptions like underflow or overflow, instructionexceptions like division by zero, or panic events like resourcedepletion or time outs. Because the routing network 320 is able to benegatively affected by such conditions and to provide better guaranteesfor delivery and handling of critical events, there is a separatecommunication network 330 for such events that is connected by abidirectional bus 124 to the controller 120. The bidirectional event bus124 is used by the processor fabric 160 and the controller 120 tocollect, aggregate, and steer the computation that is evolving insidethe processor fabric 160. The structure of the processing element 310 isdepicted in FIG. 4 . Before the start of a computation in the processorfabric 160, the controller 120 writes control information regarding thesingle assignment program into the PE's program store 440. Theinformation written into the program store 440 includes an identifier todisambiguate recurrence equations, a domain of computation on which therecurrence equation is active, a signature update program, and a routingvector. The domain of computation is a specification, typically aconstraint set defined by a system of inequalities. An example of such aconstraint set is {(i,j)|1≤i,j≤N}. This would be the domain ofcomputation of the recurrence equation shown in FIG. 7 . This constraintset is a system of four inequalities:

-   -   1. i≥1    -   2. j≥1    -   3. i≤N    -   4. j≤N

This system of inequalities can be described by the following matrix:

${\begin{pmatrix}{- 1} & 0 & 1 & 0 \\0 & {- 1} & 0 & 1\end{pmatrix}\begin{pmatrix}i \\j\end{pmatrix}} \leq \begin{pmatrix}{- 1} \\{- 1} \\N \\N\end{pmatrix}$

This constraint matrix and right hand side vector can be used as thenormalized form to specify the constraint set for some embodiments. Thisprogram information is delivered to the PEs 310 through control packetsthat are injected into the processor fabric 160 by the controller 120.The port arbiter 410 of each PE 310 receives packets from the routingnetwork 320. The port arbiter 410 selects one or more packets from thenetwork ports of the PE 310 and forwards these packets to the packetdecoder 420. The packet decoder 420 inspects the packet to determine ifit contains programming or other type of control information. If thepacket is a control packet, the packet decoder 420 sends the packet tothe controller 430 which extracts the control information and programsthe different elements of the data path, in particular, the programstore 440 and the instruction store 450. The control and statusinformation is written via the control bus 431. The program store 440will receive for each recurrence equation that is part of the singleassignment program, an identifier, a specification of a domain ofcomputation, a signature update specification, and a routing vector.This information defines some affine recurrence equation which the PE310 will help execute In general, the recurrence equation executes onmultiple PEs 310, so it is more natural to state that the recurrenceequation executes on the processor fabric 160. After the controller 120is done programming the processor fabric 160, execution is able tocommence. The execution starts with the data streamers 140 injecting thefirst data packets into the processor fabric 160 (via the crossbar 150)When data packets arrive on network ports of a PE 310, the packetarbiter 410 selects one or more packets and forwards them to the packetdecoder 420. The packet decoder 420 inspects the packet to determine ifit is a data packet that belongs to a computation that executes on thatparticular PE 310. If it does, the packet decoder 420 extracts therouting vector of the packet. If the routing vector is not null, thepacket decoder 420 forwards the packet to the packet router 425. Thepacket router 425 computes the next leg in the route, updates therouting vector of the packet, and presents the updated packet to theport arbiter 410 to be injected back in the routing network 320. If therouting vector is null then the packet decoder 420 sends the data tokento the instruction store 450. The instruction store 450 extracts theinstruction tag from the data token and assigns the data payload to theappropriate operand slot in the associated instruction stored andpending in the instruction store 450, or it allocates a new instructionif this is the first data token received for this particularcomputational event. When a pending instruction has received all itsoperands, the instruction store 450 will deallocate the instruction fromthe pending instruction list and queue the instruction token forexecution by sending it to the token disassembly unit 460. Theinstruction token includes the instruction opcode, the variableidentifier of the recurrence equation, the signature of thecomputational event this instruction represents, and the constituentoperands. The token disassembly unit 460 extracts the signature from theinstruction token, and sends the signature with the variable identifierto the signature pipeline 470. The signature pipeline 470 looks up thevariable identifier in the program store 440 to retrieve the signatureupdate program to apply to the signature. The signature update programis a simple affine transformation on the incoming signature, which asindicated in the general description section, is able to be interpretedas a spatial index vector in some abstract lattice. The signaturepipeline 470 applies this aftine transformation to the incomingsignature to produce a new signature. This new signature is forwarded tothe token assembly unit 490. Concurrently with the execution of thesignature pipeline, the value pipeline 480 executes the instruction togenerate a new left hand side value. The token disassembly unit 460extracts the instruction opcode and operands from the instruction tokenand forwards that to the value pipeline 480. The value pipeline 480executes the instruction and forwards the result to the token assemblyunit 490. The token assembly unit 490 takes the output of the signaturepipeline 470 and the output of the value pipeline 480 and constructs anew data token. It checks the signature of this new data token againstthe domain of computation for this recurrence equation, and if insidethe domain, it sends the data token to the packet router 425. If therouting vector is not the null vector, the packet router 425 embeds thedata token into a packet and forwards that to the port arbiter 410 to beinjected back into the routing network 320 under the control of somearbitration policy. Examples are first-come-first-served, or prioritybased schemes to implement quality-of-service guarantees. If the routingvector of the data token is null, it implies that the data token isrecirculating inside the current PE 310 and the packet router 425 sendsthe data token to the instruction store 450 where it is matched up withan instruction.

Referring to FIG. 5A, there is shown one possible embodiment of thestructure of data packets 510 flowing through the routing network 320.Data packets 510 flow through the routing network to deliver data valuesto the PEs 310. In this embodiment, the data packet 510 includes apacket identifier 511, a queue identifier 519, a stream identifier 512,a recurrence equation identifier 513, a data type 514, a signature 515,a data value 516, and a routing vector 517. The packet identifier 511 isa debug feature to help identify any computational errors. During theexecution of a single assignment program on the fabric, thousands ofpackets are able to be in the system at any point in time. To be able toidentify a specific packet in this collection requires that a uniqueidentifier is carried. This information would be akin to debuginformation and instructions that are injected in the instruction streamof an instruction stream processor by its compiler to help debugfunctional problems in the code. During execution of release code, thisunique packet identifier would not be part of the data packet.

Still referring to FIG. 5A, this exemplary data packet 510 is able to beused in a 2-dimensional processor fabric executing 4-dimensional singleassignment programs. The 2-dimensional processor fabric would determinethat the routing vector 517 is 2-dimensional and represents a Manhattanroute to the destination PE 310. As described in the operation of the PE310 a non-null routing vector would be detected by the packet decoder420 and forwarded to the packet router 425 to be routed to the next PE310 in the route. The 4-dimensional single assignment program wouldmanifest itself in the signature 515 to be a 4-vector representing the 4indices of 4-dimensional space. The signature 515 combined with therecurrence identifier 513 are very important in program executionaccording to the present invention. The recurrence identifier 513 andsignature 515 uniquely identify one computational event in the singleassignment program. The result of that computational event is stored inthe data value slot 516, and its type is identified by the data type514. To deliver the result to the next computational event itparticipates in, the network routes the data packet via the routingvector 517, and uses the queue identifier 519 to help the packet routers425 in the PEs 310. Each recurrence equation gets assigned a uniquequeue ID by the compiler, which enables the packet routers to quicklyqueue incoming data packets with minimum decode hardware. Given the factthat in a fully active 2D routing mesh there are four concurrentincoming packets, the pressure on the front-end of the PE 310 issignificant By carrying some of the queue information in the datapacket, the hardware complexity of the packet router in the PE 310 isreduced. Finally, the stream identifier 512 is used to associate a datapacket to a flat memory data structure. When the data streamers 140generate multidimensional data streams from a flat memory datastructure, the signature identifies the location within the datastructure but the stream identifier 512 is needed to identify the datastructure. This is akin to base addresses used in instruction streamprocessors where the program that executes on a memory data structure isspecified relative to a base address. The stream identifier 512 enablesthe data streamers to properly identify where the data packet comes fromand where it is going to in memory.

Now referring to FIG. 5B, there is shown one possible embodiment of thestructure of a data token 520 that would be used with the data packetdepicted in FIG. 5A. Data tokens 520 carry the minimum informationneeded for the instruction store 450 to identify which instruction thisdata element belongs to. As is able to be seen by comparing FIG. 5A andFIG. 5B, the data token 520 is completely contained within the datapacket 510. The individual fields in the data token are the same fieldsas described in the data packet description above. This is a commonstructure among different embodiments since the data packets are reallyrouting mechanisms to deliver data tokens throughout the processorfabric 160. By using the same structure between data packets and datatokens, the data tokens are able to be quickly assembled anddisassembled, which reduces hardware complexity and improves performanceboth in terms of power as well as throughput and latency. The structureof the data token determines the detailed operation of the instructionstore 450, which in many ways is able to be seen as the core controlmechanism that enables execution of arbitrary single assignment programsarising from systems of affine recurrence equations. The instructionstore 450 organizes data tokens and assembles and organizes instructiontokens pending execution. The instruction store 450 embodiment thatbelongs with the data token structure 520 as depicted in FIG. 5B isshown in FIG. 6 .

Referring to FIG. 6 , data tokens arrive at the data token disassemblyunit 610. The data token disassembly unit 610 extracts the differentfields of the data token. To identify the instruction to which this datatoken must be delivered, the data token disassembly unit 610 extractsthe recurrence identifier 513 and the signature 515 from the data tokenand sends that to a lookup unit 620. The lookup unit 620 queries theprogram store 440, shown in FIG. 4 to retrieve the instructioninformation. The lookup unit 620 constructs an instruction tag 625 bysimply combining the recurrence identifier 513 and the signature 515.This is a unique identifier for the computational event that this datatoken participates in. Additionally, the lookup unit 620 also receivesinformation from the program store 440 about the slot location the datatoken occupies in the instruction. This information controls the slotassignment unit 630, which receives the data type 514 and the raw datavalue 516 from the data token disassembly unit and routes this to theappropriate slot in the pending instruction. The last function thelookup unit 620 performs is the construction of an instruction header651, which contains information about how to manage the pendinginstruction in the pending instruction token store 650. Now referringback to FIG. 5C, there is shown an embodiment of an instruction tokenthat is managed in the pending instruction token store 650. Theinstruction header 651 includes the instruction opcode 531, the slotcover 532, and the slot occupancy field 533. The instruction opcode 531is a field that controls the functional units in the value pipeline 480.It encodes the type of operation that needs to be applied to theoperands. Examples are the typical functional unit operators such asADD, MULTIPLY or DIVIDE, or any logic operations such as MASK, BIT_TEST,or SHIFT The slot cover field 532 specifies how many operands theinstruction requires. For example, for simple instructions, a singleinput operand is able to be used whereas for more complex instructions,a typical three operand structure is used. The slot cover field 532 andthe slot occupancy field 533 work together in the pending instructiontoken store to determine if an instruction has received all itsoperands. The process that is used starts with the lookup unit 620composing an instruction tag 625 and sending it to the tag CAM 640. Thetag CAM 640 is a content addressable memory that is addressed by theseinstruction tags. When the tag CAM 640 indicates a miss, this impliesthat the instruction has not been yet allocated in the tag CAM 640, andon a miss, the tag CAM 640 will allocate a pending instruction. If thetag CAM 640 indicates a hit, this implies that there is a pendinginstruction to which the data token is able to be delivered. The tag CAM640 provides a pointer into the pending instruction token store 650 tothe storage associated with this pending instruction. The lookup unit620 provides the instruction header 651 as previously described, and theslot assignment unit writes the data token value into the appropriateslot in the pending instruction. Furthermore, it will update the slotoccupancy field 533 and compare it to the required slot cover 532. Ifthe two are equal, all operands have been received and the instructionis ready for execution. The instruction tag 625 and the pendinginstruction 645 including fields 651, 534, 535, and 536, are send to theinstruction token assembly unit 660, which will construct theinstruction token 530 as shown in FIG. 5C The instruction token assemblyunit 660 will forward completed instruction tokens to the execute units,starting with the token disassembly unit 460, as depicted in FIG. 4 .

FIG. 8 illustrates a flowchart of a method of utilizing an executionengine. In the step 800, a program is requested from a memory by acontroller including presenting a read request to a memory controllerwhich translates the read request to a memory request and returns datato the controller. The data contains program instructions to execute asingle assignment program. In the step 802, the program instructions aredecoded into programming information and translated and delivered todata streamers. In the step 804, the programming information is sent toa crossbar. In the step 806, the programming information is deliveredfrom the crossbar to a processor fabric. In the step 808, theprogramming information is read, and the processor fabric is configuredto execute the single assignment program. In the step 810, an event isissued from the controller to the streamers to start sending datastreams once an overall data path is set up. In the step 812, read andwrite requests are received from the streamers, and the read and writerequests are transformed into memory requests at the memory controller.In the step 814, data streams are routed by the crossbar to rows orcolumns in the processor fabric for executing instructions on thestreams and producing output data streams, wherein the output datastreams are written to the memory by traversing to the crossbar to thestreamers that associate memory addresses to the data streams and thenpresent the data streams to the memory controller which writes the datastreams into the memory. In the step 816, notifier events are sent tothe controller once the streamers identify completion of a computationaltask.

To utilize the execution engine, a user inputs and/or initiates aprogram to the execution engine, for example if the execution engine isimplemented in a computer. The execution engine then executes theprogram as described above. Depending on the program, the programoutputs a desired result. For example, if the user wants to perform acomputationally complex mathematical equation, the output after theexecution engine executes is the result of the equation.

In operation, by organizing the execution of the single assignmentprogram in the form described herein, the execution engine has solvedmany problems encountered when using an instruction sequence processor.There is no instruction pointer that guides the execution. The executionengine is completely data driven. When data elements become availablethey trigger dependent computations Thus, the execution engine functionslike a data flow machine. However, in a data flow machine, there is nospatial structure to take advantage of since the lookup is done on thebasis of flat memory addresses. The second problem with the traditionaldata flow machine is that the CAM that holds the pending instructionsneeds to be very large to be able to execute a large scale program.Content addressable memories become less power efficient and slower whenthey become larger. The CAM has been the traditional bottleneck in adata flow machine because it cannot compete on performance with aninstruction sequence processor using a von Neumann architecture. Theexecution engine includes spatial constraints added to the specificationof the computation that the architecture honors, and thus energyconstraints are able to be be captured in the program. Furthermore, thespatial constraints allow distribution of the CAM across all theprocessing elements, and thus the architecture scales again in terms ofconcurrency. As an example, a small instance of the execution engine isable to integrate 4096 processing elements on a single chip Eachinstruction store in the processing elements could contain 64 pendinginstructions for a total concurrency of 262144 instructions in a singlechip. Typical concurrency measures in a 4 core chip multi processor areof the order of 100, and even highly parallel 256 processor graphicsprocessing units are limited to concurrency measures of the order of10000. With the execution engine, the ability to manage vasts amount ofconcurrency is unparalleled.

Core Engine

The basic execution method for domain flow programs, which are expressedas systems of affine recurrence equations is described herein. Therecurrence is a mechanism to express the evolution of a complexcalculation. Furthermore, the placement of the individual computationalevents in an abstract index space allows an algorithm designer toincorporate space and time constraints in the algorithmic expression.The nature of the Knowledge Processing Unit (KPU) core engine is that itallows the faithful execution of those fine-grain space and timeconstraints even under conditions of resource contention. The resourcecontention resolution of a von Neumann machine uses Random Access Memoryto serialize intermediate results so that a computational sequence canshare computational resources. In contrast, the KPU core enginedescribed herein provides a bounded physical, but logically infinitecomputational fabric, and maps the recurrence on a path of computationalresources in the fabric. Thus, the core engine represents the apparatuswhich can execute these systems of affine recurrence equations, honoringspace-time constraints by interpreting the recurrence indices aslocations in space-time, and the affine dependencies as physicalcommunications between computational events which are mapped to afunctional unit in the fabric. The application of this new method ofexecution for fine-grain parallel algorithms provides new opportunitiesto refine and optimize the method and apparatus for broader, or morespecialized applications.

Instruction Set Architecture, or ISA

A typical system of recurrence equations will use a very small set ofoperators, that might radically differ between algorithms. For example,an algorithm for a Finite Difference Method, or FDM, might only needfloating point ADDs and MULTs, potentially at different accuracies,whereas an algorithm for a Bloom filter for genomic read sequencerenormalization or assembly, requires an instruction set includinghashing functions over strings of a 2-bit alphabet. The energydissipation between these two instruction sets would be wildlydifferent, thus providing opportunities for value creation byspecializing the ISA of the KPU tailored to classes of algorithms. Inparticular, ISAs centered around hashing would do well ingenomics/proteomics, but hashing is also key in security applications,and database applications. For FDM, FEM, and FVM, instruction setsoptimized for interpolations and resampling would optimize performanceper Watt. Signal processing, image processing, and sensor processingwould all benefit from custom ISAs.

Input/Output Path

As the KPU provides better performance per Watt as compared tomulti-core and many-core Stored Program Machines, or SPMs, it willfrequently be used for real-time processing of signal processing paths.Power efficiency is important in these applications, as is the abilityto match processing time with data input rates. This means that therewill be opportunities to optimize the data path between sensors ornetworks or devices to directly feed into the streamers of the KPU.Secondly, the streamers also orchestrate the serialization of input,intermediate, and output data to and from Random Access Memory, or RAM.When this RAM is of the dynamic type, that is DRAM, access patterns ofsaid DRAM favor page locality. This provides an opportunity to imbue thestreamers with page awareness, so that caching is able to be used toaccumulate page coherent data for more efficient writeback to DRAM, orfor more efficient input stream generation from DRAM to fabric streams.Thirdly, for certain data structures, in particular, vector andmatrices, dense and sparse, as well as trees and lists, the structure ofthose data structures provide a rich ground for micro-architectureoptimization. For example, a matrix descriptor is used by the streamersto derive data streams to and from memory. For sparse matrices, specialindex structures are used, to minimize memory bandwidth and thusmaximize performance for a given DRAM technology. Fourthly, when we havespecific functional unit micro-architectures, such as SIMD units thatcan do four floating point operations per instruction, the data streamshould match that capability. This means that the streamers work inconcert with the micro-architecture of the functional units. SIMDfloating point units are one example, string and hash operators in textor genome/proteome algorithms would also provide new opportunities foroptimizations.

Algorithms

As each algorithm needs to be transformed into a system of affinerecurrence equations, and clever spatial placements and alignments ofthe domains are important for good performance, each new domain flowalgorithm has the potential to be a method and apparatus to solve thatproblem. Since a domain flow program really is a physical embedding, itrepresents an apparatus. Intricate space and time constraints could makeone system of recurrence work well and another, functionally the same,perform poorly. Secondly, algorithms would also combine with ISAs andI/O optimizations for very unique and optimized machines.

Compilation and Scheduling Optimizations

Given a system of affine recurrence equations, the compiler analyzes theamount and structure of the concurrency inherent in the algorithm. Thisis used to select good time schedules and low contention spatialallocation on the fabric. Simple algorithms are able to be used toselect space and time projections. The core algorithms are able to behighly parallel and allow selection of simple linear combinations of thedependency vectors. More complicated, phased execution patterns are ableto be used, for which these simple methods are not sufficient forcompletely automated compilation.

Processing Elements (PEs)

The PEs of the KPU are small processors including a CAM, an instructionscheduling/dispatch queue, one or more functional units, and a routerthat is able to generate affine routing vectors. The PEmicro-architecture has many opportunities for continued innovation. Forexample, clever SIMD functional units for floating point, integer, andstring operations, and instruction chaining where results are forwardedto other functional units without the need to go through the CAM orrouter. This is particularly interesting for composite instructions,such as, hashing functions, linear interpolations, or lerps, and otherhigher order functions useful in FVM, FEM, FDM, and BEM. Discrete eventmathematics such as Lattice Boltzmann Methods, or complex approximationtechniques used in FMM, are interesting methods and machines.

Processing Fabric

The processing fabric can exhibit global functional operators, such asbroadcasts, and reductions, such as Fetch-and-Add and similar methods.These global operators would be driven by instruction sets that aremanaged and maintained by the fabric, not the individual processingelements. Secondly, as the ability to integrate larger and largerfabrics with potentially tens of thousands of processing elements, theneed for fault tolerance of processing elements and the fabricincreases. Since an important aspect of the core KPU engine is thatlocally, fine-grain dependencies are invariant to the resourcecontention demands of the input data set, dealing with PE failures isgoing to be difficult and thus solutions to the fault tolerance problemwould be significant innovations. BIST testing and reconfigurabilitytechniques to identify, and isolate, faulty processing or storageelements is applicable to the KPU chips. In some implementations,instead of one big KPU fabric, a chip is made up of many, smallerfabrics that is able to efficiently communicate data streams to eachother. This would also be a micro-architecture organization to optimizemultiple communicating kernels where each kernel would optimize space,time, and ISAs for the fabric and processing elements.

Program Storage and Management

The density of a domain flow program is very high. That is, very largeand intricate parallel computations can be expressed in a hundred bytesor less. As more complex applications would chain multiple of suchparallel kernels, the micro-architecture optimizes to chain and cachethese program descriptions. Caching would be used to efficiently recalla previous kernel, and chaining would be used to avoid having toserialize intermediate data to and from memory, thus improvingperformance.

Circuits

Power management techniques that identify idle subsets of the fabriccould shut down these resources to lower operating power consumption.Secondly, the core KPU engine is a data driven engine, so it could beimplemented as a pure asynchronous execution pipeline.Micro-architectural implementations around signal settling MullerC-element pipelines or micro-pipelines could offer low powerimplementations of the KPU in embedded applications.

Field Programmable Gate Arrays

As a domain flow program concept provides an opportunity to optimize theISA of the processing elements and the fabric, FPGA implementations ofthe KPU would make it possible to switch fabrics and instruction setsduring the execution of the application. This could lead toworkload-optimized servers and embedded processors.

The present invention has been described in terms of specificembodiments incorporating details to facilitate the understanding ofprinciples of construction and operation of the invention. Suchreference herein to specific embodiments and details thereof is notintended to limit the scope of the claims appended hereto. It will bereadily apparent to one skilled in the art that other variousmodifications may be made in the embodiment chosen for illustrationwithout departing from the spirit and scope of the invention as definedby the claims.

1. A computing device comprising: a memory for storing data and a domainflow program; a controller for requesting the data and the domain flowprogram from the memory; and a processor fabric for processing the dataand the domain flow program via a plurality of processing elements. 2.(canceled)
 3. (canceled)
 4. (canceled)
 5. (canceled)
 6. (canceled) 7.(canceled)
 8. The device of claim 1 wherein for sparse matrices, indexstructures that use hierarchical blocks to enable n-bit indices are usedto minimize memory bandwidth and maximize performance for a DRAM.
 9. Thedevice of claim 1 wherein the controller is further configured forpresenting a read request to a memory controller which translates theread request to a memory request and returns the data to the controller.10. (canceled)
 11. (canceled)
 12. (canceled)
 13. The device of claim 1further comprising a crossbar configured for routing the data to rows orcolumns in the processor fabric.
 14. The device of claim 13 wherein theprocessor fabric is further configured for producing output data. 15.The device of claim 14 wherein the output data is written to the memoryby traversing through the crossbar according to a data structuredescriptor definition and whereupon the data are is presented to amemory controller which writes the data into the memory. 16-30.(canceled)
 31. The device of claim 1 wherein the processor fabric isprogrammable.
 32. The device of claim 1 wherein the processor fabricreceives the data, executes instructions on the data, and producesoutput data.
 33. The device of claim 1 wherein the data comprisesmultidimensional data and is processed under control of the domain flowprogram which is a single assignment program.
 34. The device of claim 33wherein the plurality of processing elements comprises a regular networkof processing elements capable of executing the single assignmentprogram.
 35. The device of claim 1 wherein the plurality of processingelements comprise a matrix of processing elements.
 36. A methodcomprising: storing, in a memory of a device, data and a domain flowprogram; requesting, with a controller, the data and the domain flowprogram from the memory; and processing, with a processor fabric, thedata and the domain flow program via a plurality of elements.
 37. Themethod of claim 36 wherein for sparse matrices, index structures thatuse hierarchical blocks to enable n-bit indices are used to minimizememory bandwidth and maximize performance for a DRAM.
 38. The method ofclaim 36 wherein the controller is further configured for presenting aread request to a memory controller which translates the read request toa memory request and returns the data to the controller.
 39. The methodof claim 36 further comprising routing, with a crossbar, the data torows or columns in the processor fabric.
 40. The method of claim 39wherein the processor fabric is further configured for producing outputdata.
 41. The method of claim 40 wherein the output data is written tothe memory by traversing through the crossbar according to a datastructure descriptor definition and whereupon the data is presented to amemory controller which writes the data into the memory.
 42. The methodof claim 36 wherein the processor fabric is programmable.
 43. The methodof claim 36 wherein the processor fabric receives the data, executesinstructions on the data, and produces output data.
 44. The method ofclaim 36 wherein the data comprises multidimensional data and isprocessed under control of the domain flow program which is a singleassignment program.
 45. The method of claim 36 wherein the plurality ofelements comprise a matrix of processing elements.
 46. The method ofclaim 36 wherein the plurality of elements comprise a regular network ofprocessing elements.
 47. A processor fabric for processing domain flowprogramming information comprising: a first set of processing elements;and a second set of processing elements which communicate with the firstset of processing elements, wherein the first set of processing elementsand the second set of processing elements are configured for processingdata and a domain flow program.
 48. The processor fabric of claim 47wherein the first set of processing elements and the second set ofprocessing elements process an instruction set architecture configuredfor a specific class of algorithms.
 49. The processor fabric of claim 48wherein the specific class of algorithms comprise hashing algorithms.50. The processor fabric of claim 48 wherein the specific class ofalgorithms comprise optimizations for interpolations and resampling. 51.The processor fabric of claim 48 wherein the plurality of elementscomprise a regular network of processing elements.
 52. The processorfabric of claim 47 wherein the first set of processing elements and thesecond set of processing elements are further configured for receivingmultidimensional data and producing output data.
 53. The processorfabric of claim 52 wherein the output data is written to a memory bytraversing through a crossbar to a memory controller which writes thedata into the memory.
 54. The processor fabric of claim 47 wherein thefirst set of processing elements and the second set of processingelements recognize a spatial tag of a computational event and takeaction under control of the domain flow program.
 55. The processorfabric of claim 47 wherein the domain flow program evolves asmulti-dimensional data match up with the first set of processingelements and the second set of processing elements and produce newmulti-dimensional data.
 56. The processor fabric of claim 47 wherein thefirst set of processing elements and the second set of processingelements each include a content addressable memory, an instructionscheduling/dispatch queue, one or more functional units, and a routerconfigured to generate affine routing vectors.
 57. The processor fabricof claim 47 wherein the processor fabric manages and maintains globaloperators.
 58. The processor fabric of claim 47 wherein the processorfabric implements testing and reconfiguring to identify and isolate afaulty processing element of the first set of processing elements or thesecond set of processing elements or a faulty storage element.
 59. Theprocessor fabric of claim 47 wherein the processor fabric is implementedon a chip with one or more additional processor fabrics which are ableto communicate data to each other.
 60. The processor fabric of claim 47wherein the first set of processing elements and the second set ofprocessing elements each comprise a Single Instruction Multiple Data(SIMD) unit configured to process a plurality of operations perinstruction.